A language model adaptation approach based on text classification
نویسندگان
چکیده
In our paper, we divide the corpus into 8 domains through text classification using K-means algorithm, and calculate the trigram LMs for each one. But the experiment shows the performance in some ones becomes worse. In order to solve this problem, we try to do the LM adaptation based on the domain LMs. The adaptation is done by mixing the domain LMs with the background LM by a linear interpolation. Relative word error rate reductions of between 5 and 10 % over the pruned background LM are achieved.
منابع مشابه
A Document Weighted Approach for Gender and Age Prediction Based on Term Weight Measure
Author profiling is a text classification technique, which is used to predict the profiles of unknown text by analyzing their writing styles. Author profiles are the characteristics of the authors like gender, age, nativity language, country and educational background. The existing approaches for Author Profiling suffered from problems like high dimensionality of features and fail to capture th...
متن کاملUsing Fuzzy LR Numbers in Bayesian Text Classifier for Classifying Persian Text Documents
Text Classification is an important research field in information retrieval and text mining. The main task in text classification is to assign text documents in predefined categories based on documents’ contents and labeled-training samples. Since word detection is a difficult and time consuming task in Persian language, Bayesian text classifier is an appropriate approach to deal with different...
متن کاملUsing Fuzzy LR Numbers in Bayesian Text Classifier for Classifying Persian Text Documents
Text Classification is an important research field in information retrieval and text mining. The main task in text classification is to assign text documents in predefined categories based on documents’ contents and labeled-training samples. Since word detection is a difficult and time consuming task in Persian language, Bayesian text classifier is an appropriate approach to deal with different...
متن کاملA Summary Writing Model Based on Van Dijk’s Concept of Macrostructure and its Application within the Genre-Based Approach
This study was an attempt to provide a comprehensive model for summary writing based on the model of Van Dijk’s concept of macrostructures. The effectiveness of the model was examined in a genre-based quasi-experimental study with the data collection procedure lasting a semester. The participants included 60 female English learners divided into two experimental and control groups. The results o...
متن کاملA New Document Embedding Method for News Classification
Abstract- Text classification is one of the main tasks of natural language processing (NLP). In this task, documents are classified into pre-defined categories. There is lots of news spreading on the web. A text classifier can categorize news automatically and this facilitates and accelerates access to the news. The first step in text classification is to represent documents in a suitable way t...
متن کاملA Joint Semantic Vector Representation Model for Text Clustering and Classification
Text clustering and classification are two main tasks of text mining. Feature selection plays the key role in the quality of the clustering and classification results. Although word-based features such as term frequency-inverse document frequency (TF-IDF) vectors have been widely used in different applications, their shortcoming in capturing semantic concepts of text motivated researches to use...
متن کامل